Log file server applications are used in a distributed environment for logging network information received concurrently from distributed systems, each having its own network address. Such distributed systems may be referred to from time to time as network clients or just clients. Any kind of network information can be stored, such as network management information or client accounting data, both of which are examples of vital data. A requirement for processing and storing such vital data implies a need for a reliable log file server application. If the server application suffers an unplanned interruption, a minimum amount of data should be lost and any damaged data should be identified when the application is resumed.
The amount of data to be stored in a file server is generally a function of the size of the network. In modern networks, such as those allowing high speed inter-site communications, a large amount of data sent substantially simultaneously from a number of clients must be stored on the server. This requirement implies a need for a logging application. If the performance of the logging application is poor, significant amounts of data may be lost or degraded when multiple clients attempt to concurrently write data to the file server. On the other hand, if performance of the logging application is good, the application can use an optional reliable protocol to advise one of the clients to stop sending data. This reduces the amount of data lost.
A known solution for logging and storing network data is based on use of a multifile file server. Received data is written into a limited size file until a buffer full condition occurs for that file. A new file is then opened to provide storage of data yet to be received. This prior art solution is described in an Internet-draft document disclosing a Connection-Oriented Accounting Mib 'draft-ietf-atommib-acct-03.txt' of Keith McCloghrie, Juha Heinanen, Wedge Green and Anil Prasad, dated Jul. 31, 1996. In this document, the storage area allocated for accounting data sent by SNMP agents in a TCP/IP network has a given maximum size. When storage area is filled, the agent either automatically switches to another new file, or new records are discarded. This document encourages the use of files having at least a minimum size.
Where multiple files are employed, the prior art solution permits storage of greater numbers of records. The efficiency of data write operations are enhanced by switching to new files rather than using processing time in trying to reuse unoccupied space in the existing files. However, complex file management operations (such as assignment of a file name for a new file, closing of the existing file, opening of the new one and deleting the oldest existing file to free up storage space) are necessary before each write operation to a new file. and disk space can be fragmented over the life of multiple files. Consequently, write efficiency may be degraded over time.
The prior art proposes no mechanism for assuring the reliability of data write operations. To prevent the multiple file logging server application from introducing discrepancies into files when restarting from an unexpected interruption, backup files have to be managed. However, where use of a backup file becomes necessary, the overall performance of the data storage application can be degraded. If data is stored in multiple files and it becomes necessary to retrieve a backup copy of one of those files, there is always the possibility that the file space for that particular backup file will have been recycled to make space available for new files. The loss of a single backup file can corrupt an entire data set stored over multiple files even if the backup file space had not been reused or had been reused for a minimal amount of new data.